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Abstract 

In the current air traffic management (ATM) system, pilots and air traffic controllers have well- 
established roles and responsibilities: pilots fly aircraft and are concerned with energy 
management, fuel efficiency, and passenger comfort; controllers separate aircraft and are 
concerned with safety and management of traffic flows. Despite having different goals and 
obligations, both groups must be able to effectively communicate and interact with each other for 
the ATM system to work. This interaction will become even more challenging as traffic volume 
increases dramatically in the near future. To accommodate this increase, by 2025 the national air 
transportation system in the U.S. will go through a transformation that will modernize the ATM 
system and make it safer, more effective, and more efficient. This new system, NextGen, will 
change how pilots and controllers perform their tasks by incorporating advanced technologies 
and employing new procedures. It will also distribute responsibility between pilots, controllers 
and automation over such tasks as maintaining aircraft separation. The present chapter describes 
three plausible concepts of operations that allocate different ATM responsibilities to these 
groups. We describe how each concept changes the role of each operator and the types of 
decisions and actions performed by them. 
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Factors Influencing the Decisions and Actions of Pilots and Air Traffic Controllers 
In Three Plausible NextGen Environments 

The word culture is often used to refer to shared ways of thinking and behaving among 
members of any group (Kurosu & Hashizume, 2009). Cultures are also made up of subgroups, 
each with its own unique roles and obligations. Differences can thus arise between subgroups, as 
each one will have its own shared values, roles, and responsibilities. Cultures can change 
through time as the roles and responsibilities of the subgroups are altered. In this chapter, we 
explore differences between two professional groups of human operators in the air traffic 
management (ATM) system — pilots and controllers. We examine how their interactions change 
when the current modes of operation are altered such that the pilots and controllers have to share 
responsibility for separation assurance with each other or with automation. 

Air transportation is one of the safest modes of transportation, resulting in only 2 fatal 
accidents in 2008 for U.S. Air Carriers (operating under 14 CFR 121; National Transportation 
Safety Board - Aviation Branch, 2010) compared to 34,017 motor vehicle fatal crashes (National 
Highway Traffic Safety Administration, 2010) for that same year. Although the topic of safety is 
a concern to all operators in the ATM system, context and occupational culture are factors that 
influence decision-making and risk perception of operators. Mearns, Flin, and O’Connor (2001) 
studied pilots engaging in crew resource management training and noted, “Sources of conflict 
arise where different professional or occupational cultures clash in the view of how the workload 
should be managed, who should make decisions or how the situation should be resolved” (p. 

378). This quote captures the importance of examining the roles and responsibilities of operators 
within a system. Understanding how differences in professional cultures influence operator 
decision-making and action is even more important when new technologies or procedures are 
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being introduced because these technologies change the roles and responsibilities of the 
operators employing them. 

The impressive safety record of the National Airspace System is achieved in the current- 
day ATM system through a centralized system that is ground-based and human-intensive. The 
primary operators in this system are those of air transport pilots (ATPs) and air traffic controllers 
(ATCs). As shown in Table 1, the roles and responsibilities, activities, primary displays and 
training requirements for the two types of operator are very different. ATCs are responsible for 
the safe and expeditious movement of traffic. ATCs are trained on flight rules, air traffic 
management procedures, weather, communications and operational procedures that allow them 
to plan the movement of aircraft and transmit instructions to ATPs for carrying out their 
clearances. Because of the lateral separation precision needed to do their job, ATCs rely on a 
two-dimensional radar display that is best suited to show aircraft locations in their airspace. 
ATPs, on the other hand, are responsible for transporting passengers and cargo in a safe and 
expeditious manner. ATPs process instructions received from the ATC and act on them. The 
primary flight deck displays are designed for vertical profile planning, fuel management, and 
meeting flight scheduling requirements (Kerns, 1999). ATP training involves basic airmanship, 
aircraft systems operation, and navigation. As this brief description indicates, the primary 
operators in the National Airspace System use different procedures to accomplish their tasks, and 
the type of information and training they are given are not designed to achieve common tasks. 
Note also that the primary means of communication and coordination is through voice. A 
formalized language (“standard phraseology”) was developed for this purpose, but in busy traffic 
situations, voice communications may not follow these standard operating procedures. 
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Air Traffic Controller 

Air Transport Pilot 

Roles and Responsibilities 

Roles and Responsibilities 

• Responsible for safe and expeditious 

Managing Flight Deck Systems to: 

movement of air traffic 


• Pivotal role is planning movement of 

• Transport people and cargo in a safe, 

air traffic and transmitting instructions 

efficient manner 

to carry out plan 

• Role is processing controller advisory 
information, accepting and acting on 
controller instructions 

• Managing relative to environmental 
conditions 

Primary Display Characteristics 

Primary Display Characteristics 

• Two dimensional display with 

• Flight management and display systems 

directional traffic symbols with data 

support lateral and vertical profile 

tags (speed and altitude) for 3- 

planning in all phases of flight, fuel 

dimensional radar separation 

management, system assessment and 

procedures and representations of 

flight schedule management 

vector solutions to separation and 
ascending/descending spacing profiles 


Major Training Topics 

Major Training Topics 

• Basic flight, weather 

• Ground school basic flight (Visual 

• Flight rules, traffic control procedures 

Flight Rules, Instrument Flight Rules, 

and management, and sector specific 

Parts 91 & 121) 

information 

• Basic airmanship, aircraft systems 
operation, navigation, weather 


Table 1. Differences between current-day air traffic controllers and air transport pilots in roles and 
responsibilities, primary tasks and primary displays, and training requirements (adapted from Kern, 
1999; Pohlman & Fletcher, 1999). 


The Next Generation Air Transportation System (NextGen) is a nation-wide 
transformation of the ATM system in the U.S. that will modernize the system and increase its 
capacity (Joint Planning and Development Office, JPDO, 2010). To achieve these goals, 
NextGen will use advanced technologies such as (a) controller-pilot datalink communications, 
(b) cockpit displays of traffic, weather, and terrain, (c) conflict alerting and resolution tools, and 
(d) semi-autonomous automated agents. It will also involve adopting new procedures and 
operating concepts such as trajectory-oriented operations and performance-based navigation 
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procedures that will optimize the traffic flow (see, e.g., National Aeronautics and Space 
Administration, NASA, 2007). NextGen is expected to benefit everyone. The Federal Aviation 
Administration (FAA, 2010) estimates that by 2018, the NextGen program will reduce delays by 
21%, save 1.4 billion cumulative gallons of fuel and eliminate 14 million tons of CO? emissions. 
However, for NextGen to fulfill its promise, its concepts of operation and technologies must be 
evaluated for their effect on human operator performance, as these operators must interact safely 
and effectively within the system. 

One important cultural change expected from NextGen is distributed decision-making. 
Critical decisions that are currently the responsibility of the ATC will be made at a more local 
level by the operator with the most dependable information. Coordination between all operators 
in the system will be facilitated by real-time and shared information regarding traffic, weather, 
and aircraft intent. Because of the revolutionary changes being brought about by NextGen, it is 
how operator performance will change with the adoption of specific concepts and technologies 
need to be determined. Current-day ATC performance is measured in terms of safety and 
efficiency (e.g., Rantanen, 2004). Safety metrics include number of incidents and violations 
(e.g., losses of separation) and accidents (e.g., fatal vs. non fatal); efficiency metrics include 
average delays, and distance/time travelled per aircraft (e.g., Pierce, Vu, & Strybel, 2008). ATP 
performance is often measured by aircraft state (e.g., airspeed, descent rate, and glide slope; 
Gawron, 2000). However, both ATC and ATP performance is affected by the interaction of 
system and cognitive factors, and the cognitive constructs must be measured to evaluate how 
they are affected by the potential NextGen concepts of operation. Two of these critical operator 
cognitive factors are situation awareness and workload. 



Factors Influencing Decisions— 7 


Situation awareness can be defined as an operator’s “perception of the elements in the 
environment. . .the comprehension of their meaning, and the projection of their status in the near 
future” (Endsley, 1995, p. 36). Situation awareness is particularly important to operators who 
work in a complex system where the environment is constantly changing and where there is a 
great deal of information to keep track of in order to anticipate future events (Durso & Gronlund, 
1999). Poor situation awareness can lead to decisions that have fatal consequences. For 
example, Rodgers, Mogford, and Strauch (2000) examined ATC errors and found that the origins 
of many of the errors were in failures of situation awareness (see also Jones & Endsley, 1996). 
Although most of these studies are of pilot error in the U.S.A., failures of situation awareness 
have also been tied to operator error in Eastern cultures, such as India (e.g., Kumar & Malik, 
2003). 

Mental workload refers the cognitive demands placed on an operator relative to the 
operator’s processing capacity (Hart & Staveland, 1988). Although situation awareness is 
considered to be a separate construct from mental workload, it is related to workload (Pierce, 
Strybel, & Vu, 2008): changes in workload can affect situation awareness and vice-versa. 

Jentsch, Barnett, Bowers, and Salas (1999), for example, searched NASA’s Aviation Safety 
Reporting System (ASRS) database for aviation incidents in which loss of situation awareness 
was known to be a contributing factor. They identified 221 such cases and found that in 142 of 
them, the captain was flying the aircraft and in 79 the first officer was flying the aircraft. Jentsch 
et al. (1999) also found that the captain exhibited lower situation awareness and made more 
errors when s/he was flying the aircraft during critical periods compared to when the first officer 
was flying the aircraft. They attributed the captain’s lower level of situation awareness to the 
additional workload produced by flying the plane while simultaneously engaging in critical, 
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decision-making activities. In other words, the captain’s capacity to engage in cognitively 
demanding, decision-making tasks was reduced by the increase in workload associated with 
flying the plane. 

Because NextGen will employ automation to help operators perform tasks that they may 
not have the capacity to do currently (see Prevot et ah, 2009), the effect of automation on 
operator situation awareness, and workload must be evaluated. The costs and benefits of 
automation have been explored in aviation and other task domains (see Parasuraman & Wickens, 
2008). Automation can reduce operator workload and allow the operator to take on additional 
roles and responsibilities; however, complete automation can also decrease the operators’ 
decision-making capabilities by reducing their situation awareness. It is known that humans do 
not perform well on vigilance tasks (Mackworth, 1969), making them poor monitors of 
automation, yet, automation often puts the human operator in a supervisory or fail-safe (i.e., 
back-up) role in which the operator is essentially performing a vigilance task (e.g., JPDO, 2007). 
Moreover, early work on NextGen automation concepts has shown that attempts to reduce 
workload may have the opposite effect. For example, in a study of pilot acceptance of automated 
conflict resolutions in 3X traffic density, Battiste et al. (2008) found that pilots wanted to contact 
a human ATC to discuss and clarify 30% of the auto-resolutions sent to the flight deck. This 
number of inquiries would greatly exceed the controllers’ capability, especially if put in a back- 
up mode. These results suggest that automating conflict resolutions for reducing ATC workload 
could create more workload if pilots are not able to interact with a human operator. 

Although, separation responsibility is currently allocated to the controller, in NextGen 
pilots can also play a critical role in ATM decision-making. Dao et al. (2009) examined the 
effect of automation level on pilot situation awareness in a laboratory task. Pilots were presented 
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short, 3 -minute scenarios in which their aircraft was in conflict with another aircraft and were 
asked to perform a conflict resolution task with or without the support of automation. On trials in 
which automation was available, pilots were able to modify proposed automated clearances in 
one block of trials, but only execute the automated clearances in another block. After resolving 
the conflict, pilots were probed for situation awareness. With the scenario frozen, but all 
displays still visible and active, pilots were asked questions relating to the scenario (i.e., about 
past events, current information displayed, or about future states if the scenario was to be 
resumed). Dao et al. found that situation awareness was lowest in the fully automated condition 
in which pilots evaluated and executed automated clearances for conflict resolution, regardless of 
the acceptability of the resolution. Situation awareness was higher when pilots created 
resolutions to the conflicts on their own or when they interacted with automation to modify 
initial clearances suggested by automation. These findings suggest that automation tools used to 
aid decision-making in conflict resolution, i.e., that leave the pilots in the loop, may be more 
effective in maintaining pilot situation awareness than the implementation of automated 
clearances. 

In the remainder of the chapter, we provide an overview of a distributed air-ground 
simulation study that examined changes in performance, workload, and situation awareness as a 
function of three strategies for traffic-separation. These strategies differed in the operator most 
responsible for separation, either human pilots, human air traffic controllers, or automation, in a 
high-density traffic environment. It has been shown that current-day air traffic controllers are 
not likely to be able to manage this level of traffic density (3x) without alleviation (Prevot et ah, 
2009). Thus, the three concepts were chosen to partially reallocate separation responsibility to 
pilots or to automation. In Concept 1, responsibility was shared between pilots and controllers, 
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with pilots assigned the majority of conflicts. In Concepts 2 and 3, responsibility was shared 
between controllers and automation, with more conflicts allocated to controllers in Concept 2 
and more to automation in Concept 3. It should be noted that these concepts of operation are not 
necessarily being endorsed by any agency as promising NextGen operational concepts. Rather, 
they were selected because they alter the decision-making characteristics of the pilots and 
controllers, and should therefore result in changes in performance, workload, and situation 
awareness. 

All scenarios for the three concepts required the pilots we were testing to accomplish two 
tasks. First, the pilots had to develop a trajectory modification that would take them around 
enroute weather systems that lay between them and their arrival airport. Second, pilots were 
required to engage in a spacing task, where they had to follow a designated lead aircraft by 105 
seconds at the merge point and to maintain that interval to the runway. Because spacing 
instructions were given before pilots modified their flight plans to go around weather, these 
modifications could make it impossible to space behind their originally assigned lead aircraft. 
Therefore, pilots sometimes needed to make subsequent requests to be re-sequenced (assigned a 
new lead aircraft) by the controllers who were responsible for re-sequencing the aircraft. 

Because separation responsibility for experimental aircraft was allocated to pilots in Concept 1 
and to automation in Concepts 2 and 3, we were able to examine how controllers responded to 
these requests as well as assess the controller’s awareness of the aircraft making these requests. 

Simulation Environment and Operator Tools 

The simulation was conducted over the Internet, distributed across four research labs: the 
Flight Deck Display Laboratory (FDDRL) at NASA Ames Research Center, the Center for 
Human Factors in Advanced Aeronautics Technologies (CHAAT) at California State University 



Factors Influencing Decisions— 1 1 


Long Beach (CSULB), the Systems Engineering Research Laboratory (SERL) at California State 
University Northridge, and the Human Integrated Systems Engineering Laboratory (HISEL) at 
Purdue University. Participant controllers were located at CSULB, and participant pilots were 
located at NASA Ames. Confederate pilots were located at all of the sites and confederate 
controllers were located at CSULB. 

The simulation was run using the Multiple Aircraft Control System (MACS) and Cockpit 
Situational Display software. MACS was developed by the Airspace Operations Laboratory at 
NASA Ames Research Center (see Prevot, 2002), and CSD was developed by the Flight Deck 
Display Research Laboratory at NASA Ames (see Granada, Dao, Wong, Johnson, & Battiste, 
2005). Both MACS and the CSD can be configured to run current-day or future distributed air- 
ground ATM operations. Both included algorithms that supported conflict detection and manual 
resolutions via pilot/controller modification of flight plans. Both the CSD and MACS also 
incorporated an auto-resolver tool, based upon the work of Erzberger (2006), that that would 
automatically generate a suggested resolution upon request from the pilot or controller. Finally, 
the conflict detection and auto-resolution capability was the core of the ground-based auto- 
resolver agent, that autonomously uplinked resolutions to some of the flight decks in Concepts 2 
and 3. 

The simulation environment mimicked Kansas City Air Route Traffic Control Center, 
Sector 90, and Indianapolis Air Route Traffic Control, Sector 91, but consisted of a larger area of 
airspace (i.e., combining a high and super-high sector to form a super sector) to load the air 
traffic controller with 3 times current day traffic. Surrounding airspace was also included in the 
scenario (called Ghost Sectors because they were not being managed by a participant controller 
but by students and staff working in the CHAAT lab). Aircraft populating the simulation were 
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designated as TFR (Trajectory Flight Rules) or IFR (Instalment Flight Rules). TFR aircraft were 
never directly managed by the human controller and, depending on the concept of operation 
being employed, may also have had conflict alerting and resolution tools (Concepts 1 and 2) or 
were capable of interacting with an auto-resolver agent to negotiate flight plan changes 
independent of a human controller’s involvement (Concepts 2 and 3). IFR aircraft were always 
managed by the human controller, had neither onboard conflict alerting nor the capability to 
interact with the auto-resolver agent. Ah experimental participants flew simulated desktop 
stations of TFR aircraft. 

The controller’s scope mimicked an Air Route Traffic Control Center (ARTCC) radar 
scope, with the active (controlled) sector highlighted. The controller radar display consisted of 
traffic and static weather. IFR aircraft were displayed at full brightness while TFR aircraft were 
low-lighted unless they were in conflict with an IFR aircraft. The weather display was static and 
depicted 2D Nexrad weather. Controllers were given the following advanced tools: conflict 
alerting (that alerts controllers about an upcoming loss of separation up to 8 minutes in advance, 
see Figure 1), trial planner (that provides controllers with a method of visually altering aircraft 
trajectory; when coupled with the conflict probe, allows controllers to create conflict-free 
trajectory changes within 8 minutes, see Figure 2). Controllers were also equipped with an auto- 
resolver tool that, at the request of the controller, brought up a suggested resolution to the 
conflict being probed. In Concepts 2 and 3 (to be described later), a proportion of the conflicts 
was delegated to an auto-resolver agent at the start of the scenario. Once it was delegated 
responsibility, the auto-resolver agent acted autonomously to resolve conflicts between 
designated aircraft (see Figure 3). Because of the high volume of traffic, several tools for 
alleviating controller workload were also included: autohandoff, auto frequency change, no radio 
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check-in for TFR aircraft (IFR aircraft performed radio checked in), and TFR radio contact only 
when discontinuing spacing or requesting conflict resolution (in an emergency). 

Pilots flew a desktop simulator with flight deck controls similar to a Boeing 757 using the 
MACS interface. Pilots also interacted with the CSD (see Figure 4), which is a 3D volumetric 
display of traffic and weather information. The CSD provided pilots with the location of 
ownship and surrounding aircraft. It also allowed the pilots the capability to view the 4D 
trajectories of ownship and all traffic (Granada, Dao, Wong, Johnson, & Battiste, 2005). The 
CSD also provided a Route Assessment Tool (RAT, Figure 5), that allowed pilots to manually 
create flight plan changes around weather. In addition, in Concepts 1 and 2 (to be described 
later), pilots were given access to advanced conflict detection and resolution tools on the CSD. 
These included 1) a conflict probe that detected and alerted pilots of an upcoming loss of 
separation, and which, when coupled with the RAT, allowed pilot generated resolutions or 
deconflictions for traffic; and 2) an auto-resolver tool (see Figure 6) that at the request of the 
pilot finds a resolution to the current conflict. When possible, more than one alternative was 
presented to the pilot (i.e., a lateral versus vertical maneuver). 

To assess situation awareness and workload, a separate touch screen tablet computer was 
used to administer online queries (probe questions to pilots and controllers). Administration of 
the probe questions followed the Situation Present Awareness Method (SPAM, Durso & Dattel, 
2004). A ready prompt appeared approximately every 3 minutes. The ready query was accepted 
by operators when they felt that the workload was reasonable enough for them to be able to read 
the question (and answer it). The time to accept a ready query can be used as a measure of 
workload because if the operator is busy at the time of the probe, it should take him or her longer 
to accept the probe. Following the acceptance of the ready query, a probe question was 
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immediately presented that asked the operator about his/her workload (workload probe) or 
situation awareness (situation awareness probe). The workload probe asked the operator to 
“Rate your workload” on a scale of 1 (low) to 5 (high). The situation awareness probes varied in 
content, from information about the conflicts (e.g., Will UPS549 be in conflict with another 
aircraft within the next 5 minutes?), sector status/traffic or effect of weather avoidance (e.g., In 
which direction did your lead aircraft deviate to avoid weather?), and 
commands/communications (e.g. Was the last command you issued a frequency change?). If the 
time to accept a ready probe does in fact measure workload, we expect those times to correlate 
with the workload ratings given for the workload probes. The time to answer the situation 
awareness probes (as well as accuracy) was our measurement of operator situation awareness, 
with faster reaction times indicating greater situation awareness. 

Concepts of Opera tion 

In Concept 1, pilots were given the primary responsibility for separation assurance. TFR 
aircraft had on-board conflict alerting, the RAT, and auto-resolver tool. IFR aircraft did not have 
on-board conflict alerting and could not interact with the auto-resolver tool; the RAT was 
available for modifying Ownship’s route for weather avoidance for both IFR and TFR aircraft. 
TFR Pilots were responsible for managing separation between TFR-TFR and TFR-IFR aircrafts 
(75% of the conflicts). For TFR-TFR conflicts, Rules of the Road were built into the system, 
where the burdened aircraft (i.e., the aircraft that has responsibility for moving) appeared in 
amber and the non-burdened aircraft appeared in blue on the CSD tool bar. For TFR-IFR 
conflicts, pilots of TFR aircraft were instructed to maneuver their aircraft; the IFR pilot was not 
responsible for resolving conflicts. The human ATC was equipped with conflict alerting, the 
trial planner, and auto-resolver tool. ATC was responsible for IFR-IFR conflicts and for 
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conflicts with any experimental aircraft on their CDA, giving the experimental arrival aircrafts 
priority. The auto-resolver agent has no responsibility. 

In Concept 2 controllers were given the primary responsibility for separation assurance. 
As in Concept 1, TFR aircraft had on-board conflict alerting, the RAT, and auto-resolver tool. 
The IFR aircraft did not have on-board conflict alerting and were not able to interact with the 
auto-resolver tool. IFR pilots were able to use the RAT to modify Ownship’s route for weather 
avoidance. TFR pilots were not responsible for managing separation, but because they have 
onboard alerting, TFR pilots could interact with auto-resolver agent and propose conflict 
resolutions for consideration by the controller/auto-resolver agent. At the beginning of the trial, 
the auto-resolver agent was allocated responsibility for managing TFR-TFR conflicts (25% of 
conflicts). The human ATC was equipped with conflict alerting, trial planner and the auto- 
resolver tool. ATC was responsible for TFR-IFR and IFR-IFR conflicts (75% of conflicts). For 
TFR-IFR conflicts, the controller was instructed to move the IFR aircraft. As in Concept 1, ATC 
was responsible for conflicts with any experimental aircraft on a continuous descent approach 
(CDA). 

In Concept 3 the automation was given the primary responsibility for separation 
assurance. Both TFR and IFR aircraft were not equipped with on-board conflict alerting. 
However, the RAT tool was available to aid rerouting for weather avoidance. TFR and IFR 
pilots were not responsible for managing separation. At the beginning of the trial, the auto- 
resolver agent was allocated responsibility for managing TFR-TFR and TFR-IFR conflicts (75% 
of conflicts). For TFR-IFR conflicts, the auto-resolver agent was programmed to move the TFR 
aircraft. The human ATC was equipped with conflict alerting, the trial planner, and auto-resolver 
tool. ATC was responsible for IFR-IFR conflicts (25% of conflicts). In addition, ATC was 
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responsible for conflicts with any experimental aircraft on their CD A, giving the experimental 
aircraft arrival priority. 

Operator Demographics and Tasks 

The simulation was run over a two week period. During each week, 8 pilots and 2 
controllers participated as test participants (referred to as experimental pilots/controllers). The 
remaining pilot and controller duties were carried out by our confederates (mainly students and 
staff working in the various labs; referred to as pseudo-pilots and ghost controllers). Due to 
some technical difficulties, though, we were not able to run all the test trials in week 1 . Thus, the 
performance data described are only from the experimental pilots and controllers from the 
second week. However, because all participants from both weeks were able to experience the 
variety of tasks and concepts, data described from the post-experimental questionnaires and 
debriefing consist of comments and ratings from all participants. 

For experimental pilots in Week 2, the professional position most recently held was 
captain by 5 of the pilots and first officer by the remaining 3. None of the participants had any 
prior experience with merging and spacing operations, and only 3 had flown a CDA. Four of the 
pilots had between 3,000-5,000 flight hours, and three had over 5,000 flight hours. All pilots had 
glass cockpit experience, with three having over 3,000 hours of glass cockpit experience. The 
experimental controllers were both retired, radar certified, ARTCC controllers. One controller 
had 34 years of experience, and one had 25 years of civilian center control experience. 

Pilots flew and controllers managed traffic in twelve, 90-minute scenarios, each concept 
of operation appearing in four of the scenarios. Again, we emphasize that these concepts of 
operation, which allocate separation responsibility across pilots, controllers and automation, are 
not necessarily being endorsed by any agency for implementation as NextGen operational 
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concepts. They were selected, however, because they should produce measurable changes in 
operator performance, workload, and situation awareness. In all scenarios, experimental pilots 
flew arrival aircraft into Louisville International-Standiford Field (SDF) Airport under one of the 
three different concepts of operation. At the start of the simulation, they were assigned a lead 
aircraft and given spacing instructions. Prior to entering sector 90, the first active sector 
controlled by an experimental controller, pilots encountered convective weather and had to 
reroute around weather by using the RAT tool. Both controllers and pilots were asked to answer 
the situation awareness and workload probes. 

Effect of Operational Concepts on Operator Decision-Making and Action 

This section summarizes the findings from the simulation in terms of how the three 
concepts of operation influenced pilot and controller decision-making and action. More details 
regarding operator performance, workload, and situation awareness under the three concepts of 
operation can be found in papers by Dao et al. (2010), Ligda et al. (2010), Vu et al. (2010) and 
Strybel et al. (2010). 

Performance 

Although there were many different performance metrics that could have been analyzed 
as an indicator of safety (e.g., violations of separation and collisions) and efficiency (e.g., 
distance traveled and fuel consumption), in this chapter we focus on loss of separation (i.e., 
aircraft within 5 nautical miles and 1000 ft of altitude) as the safety metric of interest in 
evaluating pilot and controller performance. The experimental (TFR) pilots lost separation more 
often when they were responsible for separation of Ownship from surrounding traffic in Concept 
1 (M= 0.47 LOS per run) than in Concept 2 when the human controller was responsible (M = 
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0.13 LOS per run), and in Concept 3 (M = 0.16 LOS per run), when the auto-resolver agent was 
responsible. 

Cultural differences can be seen in how pilots resolved conflicts compared to controllers. 
Although all operators tended to rely on lateral maneuvers to resolve conflicts (about 60% of 
conflicts), the ground operators (human controllers and auto-resolver agent) were more likely to 
use altitude (i.e., vertical) resolutions compared to pilots. One reason for why pilots would 
prefer to make lateral resolutions is that pilots would typically be flying at their optimal cruise 
altitude, making them more resistant to deviate from that altitude when given a choice. 
Controllers, on the other hand, would be more willing to use altitude resolutions when two 
aircraft are in conflict. Moreover, controllers were the only operators to utilize speed and 
combinations of lateral and vertical tactics to resolve conflicts. The auto-resolver tool 
implemented in the simulation did not have the algorithms to perform such resolutions. 

Pilot comments in a post-simulation questionnaire and debriefing session yielded 
qualitative information about factors that influenced their decision-making for separation 
assurance under the different concepts of operation. When given the role for separation, 
experimental pilots indicated that they needed information regarding the nature of surrounding 
traffic. This information was provided by the CSD, which aided their decision-making 
capabilities. Pilots also indicated that they wanted intent information from surrounding aircraft 
(i.e., their plans to alter a trajectory before actually executing the change). This intent feature 
was not implemented in the present simulation. Although sharing of information in real-time is 
one of the features that will be characteristic of the NextGen environments, it is acknowledged 
that there may be delays in communication due to the transmissions needed to exchange 


information. 
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In general, pilots thought all three concepts were workable in theory. Pilots were not 
apprehensive about taking responsibility for separation. However, they did want ATC 
involvement as either a back-up or a consultant for resolutions, and as a monitor to oversee the 
traffic. This finding is not unexpected given the pilots’ limited experience with resolving traffic 
conflicts. Currently, pilots are not given the responsibility for resolving conflicts, so this 
additional responsibility changes how they would respond to conflicts when they arise. Many 
pilots expressed concerns that the human controller lacked awareness of their aircraft when 
requesting help (this concern was confirmed when controllers acknowledged that they did not 
have much awareness of what the experimental aircraft were doing, as they were not given 
responsibility for TFR aircraft at the onset of the simulation). Pilots also indicated that knowing 
the goals of the surrounding aircraft was an important factor in their decision-making. Although 
the data tags of the aircraft had information about destination (which allowed pilots to project the 
general direction where the aircraft was heading and general goals of the aircraft, such as aircraft 
near their destinations are likely to descend when in conflict), many pilots indicated that 
additional information about other pilots’ goals would have been helpful as well. For example, 
one pilot indicated that s/he was a cargo pilot and was willing to go through mild weather 
because passenger comfort is not a concern, while cargo delivery time was a central concern. 
This type of information could help pilots, especially those who are engaging in self-spacing 
responsibilities to anticipate maneuvers and preferences of lead and surrounding aircraft. 

The human controllers lost separation of aircraft in their sector more often in Concept 2 
(M = 6 LOS), i.e., when they were responsible for 75% of the conflicts, than in Concepts 1 and 3 
(Ms = 2 LOS), when they were responsible for only 25% of the conflicts. Although the number 
of LOS may appear to be high (i.e., ideally LOS should be 0), we want to emphasize that the 
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traffic volume was extremely high (3x), so these numbers are not unexpected for the types of 
scenarios tested in this simulation. For example, Prevot et al. (2009) found that once traffic 
levels exceed 1.5x, controller performance decreases. The controllers exhibited different 
strategies for resolving conflicts as a function of the concept of operation. In Concepts 1 and 3, 
where they were only responsible for resolving 25% of conflicts, controllers used lateral and 
vertical tactics at a similar rate. However, in Concept 2, where they were responsible for 
resolving the majority of conflicts, controllers relied on vertical separation strategies more often 
than lateral ones. At the same time, it should be noted that these findings regarding controllers 
should be taken with caution, as only two controllers were tested and each one was managing a 
different sector. 

Similar to the pilots, the participant controllers indicated that all three concepts were 
workable in theory. However, in the post-simulation questionnaire and debriefing, they did 
identify some factors that influenced their actions and decision-making under the different 
concepts of operations. For example, in Concept 1, controllers commented on the lack of 
experience that experimental pilots had with resolving conflicts. Because pilots are not given 
this responsibility currently, this culture shift in allocation of separation assurance responsibility 
caused more workload for the controllers when pilots called in for help. Controllers also 
expressed concerns about the transitioning of responsibilities. In all concepts, the controllers 
were not generally responsible for experimental aircraft until they started their descent and were 
on their CDA. Thus, when TFR aircraft became their responsibility in “emergency” situations 
(i.e., when conflicts arose), the controllers reported that it was difficult for them to gain an 
understanding of the situation in order to help the pilots. Indeed, controllers indicated that they 
did not have much awareness of the TFR aircraft because they were not responsible for them. 
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Thus, having the controller as a back-up conflict resolver did not work well in the scenarios 
tested. 

Workload and Situation Awareness 

Pilot and controller workload and situation awareness were also measured in the 
simulation. For pilots, workload measures from four separate metrics (latency to ready prompts, 
workload ratings on workload probes, post-trial questionnaire and post-simulation questionnaire) 
all indicated that workload was relatively low and did not differ significantly between concepts 
(see Ligda et al., 2010, for more details). In contrast, workload measures from the same four 
metrics indicated that workload was moderate to high, and that it changed as a function of sector 
properties and concept of operation (see Vu et al., 2010, for more details). The controller for 
sector 90 showed higher levels of workload compared to the controller for sector 9 1 . This 
difference is related to sector properties of the simulation. Sector 90 was a larger sector and was 
affected by the weather re-routings made by pilots; the sector 90 controller was also responsible 
for re-sequencing of experimental pilots if they had to discontinue spacing. The Top of Descent 
point was also located in sector 90, which is where the pilots begin their CDA. The merge point 
for the experimental aircraft arriving into SDF was located in sector 9 1 . The differences in 
workload for the two groups of operators (pilots vs. controllers) and within a group of operators 
(e.g., controllers managing different sectors) illustrate the importance of testing the effects of 
new concepts and technologies across a range of operators and air traffic situations. 

Our situation awareness metrics (latency to situation awareness probes and accuracy to 
the probe questions) indicate that pilots were more aware of conflicts, surroundings, and 
communications/commands issued in Concept 1, when they were actively engaged in separation 
responsibility (see Dao et al., 2010, for details). However, as noted above, in Concept 1 they 
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actually lost separation more often than in Concepts 2 and 3. This indicates that situation 
awareness, although required for good decision-making (Rodgers et ah, 2000), does not ensure 
good performance. Controller situation awareness, as measured by our metrics, showed a more 
complex picture. Because we only have data from two controllers, each managing a different 
sector, we will not characterize controller situation awareness here. Rather, we note that the two 
controllers showed different levels of awareness based on the information requirements of the 
sector and concepts of operation (see Vu et ah, 2010 for a descriptive summary of their 
behavior). 

Summary and Future Research Directions 

We started this chapter by noting how cultural differences between professional groups 
can affect operator decision-making and actions. The Situation Awareness in Trajectory 
Oriented Operations with Weather simulation reported in the chapter examined how the 
performance of pilots and controllers were affected by three concepts of operations that uniquely 
allocated separation responsibility between pilots, controllers, and automation. Implementing a 
strategy where pilots or automation are primarily responsible for maintaining separation, would 
involve an important cultural shift from the present situation, where controllers are primarily 
responsible for separation assurance. Both groups of human operators indicated that all concepts 
tested were workable in theory; however, the data showed that the three concepts produced 
different effects on performance, workload, and situation awareness. Pilots showed little change 
in workload across the three concepts, but higher awareness when actively engaged in Concept 1. 
Controllers showed sector-specific changes in awareness and workload depending on concept of 
operation. 
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Because the current ATM system allocates separation responsibility to the human 
controllers, they appear to be a natural back-up to pilots and automation when the pilots are not 
able to find acceptable solutions to conflicts. Pilots used the auto-resolver tool to resolve 
conflicts when they were responsible in Concept 1 . However, they had a tendency to revert to 
controller intervention when their solutions failed (or when the auto-resolver agent did not come 
up with a resolution in a time frame comfortable for pilots). This finding of reliance on the 
human controller is consistent with the findings of Battiste et al. (2008), mentioned previously, 
where pilots wanted to coordinate with the controller on 30% of automated resolutions provided 
to them. However, when pilots called the controllers for help, the controllers were not always 
aware of the situation and thus unable to help the pilots. Controllers showed little awareness of 
aircraft for which they were not responsible from the outset, which included the experimental 
pilots in all concepts tested. Thus, the notion of using the human controller as a backup to 
automation may not work in real-life settings unless controllers can become more aware of 
pertinent information regarding these aircraft when emergency situations arise. Controllers did 
use the auto-resolver tool to bring up potential conflict resolutions, but they used the technology 
merely as a tool and not as a back-up to any decision-making fails. 

Although NextGen applies to the ATM system in the US, similar efforts are being made 
in other regions of the world (e.g., EUROCONTROL, 2010, The Single European Sky program). 
Thus, the influence of geographical and ethnic differences in decision-making should be 
explored in future studies. Li, Harris, and Chen (2007) conducted a meta-analysis of human 
factors mishaps in three regions of the world: Taiwan, India, and the United States. They 
specifically examined 18 categories of error outlined by the Human Factors Analysis and 
Classification System and found that national culture was associated with the management style 
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of the crew. Crews in the U.S., whose national culture is more individualistic and less 
authoritarian compared to Indian and Taiwanese culture, resulted in fewer aviation accidents. Li 
et al. (2007) attributed this finding to the crew members being more comfortable discussing 
problems/solutions with the captain rather than following orders. Because Li et al. noted that 
“global aviation is strongly influenced by the United States and Western Europe” (2007, p.420), 
it is important to understand the factors that affect human performance among pilot and 
controller cultures in the US and in other regions around the world. 

Future studies should also examine the leadership role of the individuals to determine the 
effect that different roles and responsibility have on decision-making, performance, and situation 
awareness. Thunholm (2009) found that military leaders reported higher levels of naturalistic 
decision-making (e.g., more spontaneous and less rational) compared to their team members and 
that this style resulted from the fact that the leader should display decisive and action-oriented 
decision-making qualities. Finally, research should be conducted to examine the effect of new 
technologies and procedures not only on current day operators, but on future ones as well. The 
ATC Strike of 1981, where the Professional Air Traffic Controllers Organization (PATCO) 
union declared a strike was considered illegal because it violated a law (5 U.S.C. (Supp. Ill 
1956) 1 1 8p. } that banned strikes by government unions. At that time, U.S. President, Ronald 
Reagan, ordered those remaining on strike to return to work within 48 hours or forfeit their jobs. 
Not all controllers returned to work and President Reagan fired 1 1,345 striking controllers 
(Wikipedia, n.d.). The FAA had to replace these controllers over the following 10 years, but 
these replacement controllers are now at or close to retirement age (currently, controllers must be 
hired prior to their 31 st birthday and retire by their 56 th birthday). 
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Another important factor to examine has to do with the subculture of student versus 
expert controllers. We have conducted a simulation study comparing these two groups, in which 
we found that highly trained students on a sector did not differ much from actual ATCs on many 
of the sector performance variables measured in the simulation (Vu et ah, 2009). However, there 
were several differences evident between students and controllers. In particular, students were 
more compliant with experimental procedures and were more willing to use new technology and 
procedures (prior knowledge did not prevent them from taking on the new roles). Students also 
reported that their SA and workload was more negatively affected by scenario difficulty than did 
controllers. Finally, students relied more on automation and alerting tools. These differences 
need to be taken in account when evaluating new technologies using current operators as they 
may not be viewed in the same way by the group that will inherit the system. 

NextGen will change how pilots and controllers perform their tasks by incorporating 
advanced technologies, employing new procedures, and having operators engage in collaborative 
decision-making. Thus it is important to evaluate cultural factors influencing the decisions and 
actions of NextGen operators with respect to their profession (e.g., pilots versus controllers), 
their roles and responsibilities, geographic region and culture, and their age group (e.g., cohort 


effects). 
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Figure Captions 

Figure 1. Illustration of conflict detection tool on the controller’s scope. Conflicted aircraft flash 
red up to 8 minutes prior to loss of separation. 

Figure 2. Illustration of the trial planner implemented on the controller’s scope. Controllers can 
click on a portal (arrow next to the call sign in the data tag) to bring up the current flight 
plan trajectory (highlighted in blue). The controller can modify the flight plan by 
clicking on the path to insert a waypoint and drag the waypoint across the scope to make 
a new path. Although not illustrated, the controller can click on the altitude in the data 
tag to select alternative altitudes for conflict resolution as well. 

Figure 3. Illustration of the autoresolver agent implemented for the ground. The autoresolver 

agent detects and sends up a resolution to the flight deck without intervention of a human 
operator. 

Figure 4. Illustration of cockpit situational display enabled with conflict detection. The system 
detects and alerts an upcoming loss of separation within 8 minutes. Conflicts are 
highlighted in amber and the data tags of conflicting aircraft are expanded. 

Figure 5. Illustration of the route assessment tool (RAT), a feature of the cockpit situational 

display that allows pilots the capability to manually created flight plan changes. When 
the RAT is used with the Autoresolver tool, the pilot can engage in conflict resolution 
and weather avoidance. 

Figure 6. Illustration of the autoresolver tool implemented in a cockpit situational display. The 
tool uses algorithms to find a resolution for conflicting traffic at the request of the pilot. 
The proposed solutions are graphically displayed to the operator. 
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